Collapsing and placement of applications

ABSTRACT

The present technology is directed to providing visibility of data flows in a multi-tier application and to help network teams understand the dataflow of an application and develop the application&#39;s dataflow. The technology is directed to an application dependency map visualized in a collapsible chart. The collapsible chart displays the policies/relationships between each logical entity that carries a multi-tier application. The collapsible multi-tier application UI displays the data flows of a multi-tier application. Such charts are large and complex, and the present technology attempts to avoid displaying the entire topology of such multi-tier applications, while focusing on dependency relationships of interest.

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No.62/171,899, entitled “SYSTEM FOR MONITORING AND MANAGING DATACENTERS,”filed Jun. 5, 2015, which is incorporated herein by reference in itsentirety.

TECHNICAL FIELD

The present technology pertains visualization of a multi-tierapplication, and more specifically to the customization of thevisualization of the multi-tier application.

BACKGROUND

Datacenters can include a large number of servers and virtual machines.As such datacenters can have a large number of data flows between eachserver and virtual machine. Monitoring and managing the network of adatacenter can be cumbersome especially with a datacenter with a largenumber of servers, virtual machines and data flows. Visualizing thenetwork of a datacenter can help network operators manage and monitorthe network of a datacenter. However, because of the large number ofdata flows, visualizing these data flows can be very cumbersome.

Multi-tier applications can be very complex. Thousands of logicalentities could potentially be responsible for the performance of anapplication. Given their complexity, a multi-tier application cannoteasily be viewed in a conventional dependency graph. Prior art solutionstoo often attempt to display an entire network structure or applicationstructure by displaying an entire chart that can be zoomed in or zoomedout (see e.g., U.S. Pat. No. 9,246,773 issued on Jan. 26, 2016). Suchsolutions rely heavily on algorithms to layout the graph. However, giventhe complexity of multi-tier applications, conventional systems are notwell suited to allowing a system administrator to explore such graphs.

BRIEF DESCRIPTION OF THE FIGURES

In order to describe the manner in which the above-recited and otheradvantages and attributes of the disclosure can be obtained, a moreparticular description of the principles briefly described above will berendered by reference to specific embodiments that are illustrated inthe appended drawings. Understanding that these drawings depict onlyexample embodiments of the disclosure and are not therefore to beconsidered to be limiting of its scope, the principles herein aredescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1 illustrates an example of a network traffic monitoring system inaccordance with some embodiments;

FIG. 2 illustrates an example of a network environment in accordancewith some embodiments;

FIG. 3 illustrates an example of a data pipeline for determiningclusters in an application dependency map in accordance with someembodiments;

FIG. 4 illustrates an example method for creating and interacting with adynamic graph in accordance with some embodiments;

FIGS. 5A and 5B illustrate example graph accordance with someembodiments; and

FIG. 6 illustrates an example system embodiment.

DESCRIPTION OF EXAMPLE EMBODIMENTS Overview

The present technology is directed to providing visibility of data flowsin a multi-tier application and to help network teams understand thedataflow of an application and develop the application's dataflow. Thetechnology is directed to an application dependency map visualized in acollapsible chart. The collapsible chart displays thepolicies/relationships between each logical entity that carries amulti-tier application. The collapsible multi-tier application UIdisplays the data flows of a multi-tier application. Such charts arelarge and complex, and the present technology attempts to avoiddisplaying the entire topology of such multi-tier applications, whilefocusing on dependency relationships of interest.

The present technology is directed to an application dependency mapvisualized in a collapsible dynamic graph. The dynamic graph displayspolicies/relationships between each logical entity of a multi-tierapplication. A graph user interface displays the dependencies betweeneach logical entity of application, and the interface allows a user toexpand or collapse a node representing a logical entity. Nodes can bemoved and anchored within the interface to allow a user to explore thegraph and find a desired view of the multi-tier application.

Edges connecting the nodes represent policies between the nodes.

The graph is created using data gathered from a data collection andanalytics layer. Data used and made visible in the collapsible tree flowchart include, e.g, data flows from one logical entity to anotherlogical entity, policies that govern the data flows from one logicalentity to another logical entity, what host the data flow came from,what host group the data flow came from; and what subnet the data flowcame from.

The UI is customizable. A user can select elements to adjust subnetgroupings and cluster groupings. Additionally the user can upload sideinformation. Examples of side information are DNS names, host names,etc.

Description

Various embodiments of the disclosure are discussed in detail below.While specific implementations are discussed, it should be understoodthat this is done for illustration purposes only. A person skilled inthe relevant art will recognize that other components and configurationsmay be used without parting from the spirit and scope of the disclosure.

The disclosed technology is directed to the visualization of data flowswithin a datacenter. Specifically, to the generation and presentation ofa parallel coordinate chart to represent analyzed data describing dataflows. The parallel coordinate chart can be used to display a largenumber of data flows in the same chart, each flow intersects a pluralityof parallel lines representing attributes of the respective flows. Byrepresenting flows in this manner it can be possible to identify outlierflows for further investigation.

Referring now to the drawings, FIG. 1 is an illustration of a networktraffic monitoring system 100 in accordance with an embodiment. Thenetwork traffic monitoring system 100 can include a configurationmanager 102, sensors 104, a collector module 106, a data mover module108, an analytics engine 110, and a presentation module 112. In FIG. 1,the analytics engine 110 is also shown in communication with out-of-banddata sources 114, third party data sources 116, and a network controller118.

The configuration manager 102 can be used to provision and maintain thesensors 104, including installing sensor software or firmware in variousnodes of a network, configuring the sensors 104, updating the sensorsoftware or firmware, among other sensor management tasks. For example,the sensors 104 can be implemented as virtual partition images (e.g.,virtual machine (VM) images or container images), and the configurationmanager 102 can distribute the images to host machines. In general, avirtual partition may be an instance of a VM, container, sandbox, orother isolated software environment. The software environment mayinclude an operating system and application software. For softwarerunning within a virtual partition, the virtual partition may appear tobe, for example, one of many servers or one of many operating systemsexecuted on a single physical server. The configuration manager 102 caninstantiate a new virtual partition or migrate an existing partition toa different physical server. The configuration manager 102 can also beused to configure the new or migrated sensor.

The configuration manager 102 can monitor the health of the sensors 104.For example, the configuration manager 102 may request for statusupdates and/or receive heartbeat messages, initiate performance tests,generate health checks, and perform other health monitoring tasks. Insome embodiments, the configuration manager 102 can also authenticatethe sensors 104. For instance, the sensors 104 can be assigned a uniqueidentifier, such as by using a one-way hash function of a sensor's basicinput/out system (BIOS) universally unique identifier (UUID) and asecret key stored by the configuration image manager 102. The UUID canbe a large number that may be difficult for a malicious sensor or otherdevice or component to guess. In some embodiments, the configurationmanager 102 can keep the sensors 104 up to date by installing the latestversions of sensor software and/or applying patches. The configurationmanager 102 can obtain these updates automatically from a local sourceor the Internet.

The sensors 104 can reside on various nodes of a network, such as avirtual partition (e.g., VM or container) 120; a hypervisor or sharedkernel managing one or more virtual partitions and/or physical servers122, an application-specific integrated circuit (ASIC) 124 of a switch,router, gateway, or other networking device, or a packet capture (pcap)126 appliance (e.g., a standalone packet monitor, a device connected toa network devices monitoring port, a device connected in series along amain trunk of a datacenter, or similar device), or other element of anetwork. The sensors 104 can monitor network traffic between nodes, andsend network traffic data and corresponding data (e.g., host data,process data, user data, etc.) to the collectors 108 for storage. Forexample, the sensors 104 can sniff packets being sent over its hosts'physical or virtual network interface card (NIC), or individualprocesses can be configured to report network traffic and correspondingdata to the sensors 104. Incorporating the sensors 104 on multiple nodesand within multiple partitions of some nodes of the network can providefor robust capture of network traffic and corresponding data from eachhop of data transmission. In some embodiments, each node of the network(e.g., VM, container, or other virtual partition 120, hypervisor, sharedkernel, or physical server 122, ASIC 124, pcap 126, etc.) includes arespective sensor 104. However, it should be understood that varioussoftware and hardware configurations can be used to implement the sensornetwork 104.

As the sensors 104 capture communications and corresponding data, theymay continuously send network traffic data to the collectors 108. Thenetwork traffic data can include metadata relating to a packet, acollection of packets, a flow, a bidirectional flow, a group of flows, asession, or a network communication of another granularity. That is, thenetwork traffic data can generally include any information describingcommunication on all layers of the Open Systems Interconnection (OSI)model. For example, the network traffic data can includesource/destination MAC address, source/destination IP address, protocol,port number, etc. In some embodiments, the network traffic data can alsoinclude summaries of network activity or other network statistics suchas number of packets, number of bytes, number of flows, bandwidth usage,response time, latency, packet loss, jitter, and other networkstatistics.

The sensors 104 can also determine additional data for each session,bidirectional flow, flow, packet, or other more granular or lessgranular network communication. The additional data can include hostand/or endpoint information, virtual partition information, sensorinformation, process information, user information, tenant information,application information, network topology, application dependencymapping, cluster information, or other information corresponding to eachflow.

In some embodiments, the sensors 104 can perform some preprocessing ofthe network traffic and corresponding data before sending the data tothe collectors 108. For example, the sensors 104 can remove extraneousor duplicative data or they can create summaries of the data (e.g.,latency, number of packets per flow, number of bytes per flow, number offlows, etc.). In some embodiments, the sensors 104 can be configured toonly capture certain types of network information and disregard therest. In some embodiments, the sensors 104 can be configured to captureonly a representative sample of packets (e.g., every 1,000th packet orother suitable sample rate) and corresponding data.

Since the sensors 104 may be located throughout the network, networktraffic and corresponding data can be collected from multiple vantagepoints or multiple perspectives in the network to provide a morecomprehensive view of network behavior. The capture of network trafficand corresponding data from multiple perspectives rather than just at asingle sensor located in the data path or in communication with acomponent in the data path, allows the data to be correlated from thevarious data sources, which may be used as additional data points by theanalytics engine 110. Further, collecting network traffic andcorresponding data from multiple points of view ensures more accuratedata is captured. For example, a conventional sensor network may belimited to sensors running on external-facing network devices (e.g.,routers, switches, network appliances, etc.) such that east-westtraffic, including VM-to-VM or container-to-container traffic on a samehost, may not be monitored. In addition, packets that are dropped beforetraversing a network device or packets containing errors may not beaccurately monitored by the conventional sensor network. The sensornetwork 104 of various embodiments substantially mitigates or eliminatesthese issues altogether by locating sensors at multiple points ofpotential failure. Moreover, the network traffic monitoring system 100can verify multiple instances of data for a flow (e.g., source endpointflow data, network device flow data, and endpoint flow data) against oneanother.

In some embodiments, the network traffic monitoring system 100 canassess a degree of accuracy of flow data sets from multiple sensors andutilize a flow data set from a single sensor determined to be the mostaccurate and/or complete. The degree of accuracy can be based on factorssuch as network topology (e.g., a sensor closer to the source may bemore likely to be more accurate than a sensor closer to thedestination), a state of a sensor or a node hosting the sensor (e.g., acompromised sensor/node may have less accurate flow data than anuncompromised sensor/node), or flow data volume (e.g., a sensorcapturing a greater number of packets for a flow may be more accuratethan a sensor capturing a smaller number of packets).

In some embodiments, the network traffic monitoring system 100 canassemble the most accurate flow data set and corresponding data frommultiple sensors. For instance, a first sensor along a data path maycapture data for a first packet of a flow but may be missing data for asecond packet of the flow while the situation is reversed for a secondsensor along the data path. The network traffic monitoring system 100can assemble data for the flow from the first packet captured by thefirst sensor and the second packet captured by the second sensor.

As discussed, the sensors 104 can send network traffic and correspondingdata to the collectors 106. In some embodiments, each sensor can beassigned to a primary collector and a secondary collector as part of ahigh availability scheme. If the primary collector fails orcommunications between the sensor and the primary collector are nototherwise possible, a sensor can send its network traffic andcorresponding data to the secondary collector. In other embodiments, thesensors 104 are not assigned specific collectors but the network trafficmonitoring system 100 can determine an optimal collector for receivingthe network traffic and corresponding data through a discovery process.In such embodiments, a sensor can change where it sends it networktraffic and corresponding data if its environments changes, such as if adefault collector fails or if the sensor is migrated to a new locationand it would be optimal for the sensor to send its data to a differentcollector. For example, it may be preferable for the sensor to send itsnetwork traffic and corresponding data on a particular path and/or to aparticular collector based on latency, shortest path, monetary cost(e.g., using private resources versus a public resources provided by apublic cloud provider), error rate, or some combination of thesefactors. In other embodiments, a sensor can send different types ofnetwork traffic and corresponding data to different collectors. Forexample, the sensor can send first network traffic and correspondingdata related to one type of process to one collector and second networktraffic and corresponding data related to another type of process toanother collector.

The collectors 106 can be any type of storage medium that can serve as arepository for the network traffic and corresponding data captured bythe sensors 104. In some embodiments, data storage for the collectors106 is located in an in-memory database, such as dashDB from IBM®,although it should be appreciated that the data storage for thecollectors 106 can be any software and/or hardware capable of providingrapid random access speeds typically used for analytics software. Invarious embodiments, the collectors 106 can utilize solid state drives,disk drives, magnetic tape drives, or a combination of the foregoingaccording to cost, responsiveness, and size requirements. Further, thecollectors 106 can utilize various database structures such as anormalized relational database or a NoSQL database, among others.

In some embodiments, the collectors 106 may only serve as networkstorage for the network traffic monitoring system 100. In suchembodiments, the network traffic monitoring system 100 can include adata mover module 108 for retrieving data from the collectors 106 andmaking the data available to network clients, such as the components ofthe analytics engine 110. In effect, the data mover module 108 can serveas a gateway for presenting network-attached storage to the networkclients. In other embodiments, the collectors 106 can perform additionalfunctions, such as organizing, summarizing, and preprocessing data. Forexample, the collectors 106 can tabulate how often packets of certainsizes or types are transmitted from different nodes of the network. Thecollectors 106 can also characterize the traffic flows going to and fromvarious nodes. In some embodiments, the collectors 106 can match packetsbased on sequence numbers, thus identifying traffic flows and connectionlinks. As it may be inefficient to retain all data indefinitely incertain circumstances, in some embodiments, the collectors 106 canperiodically replace detailed network traffic data with consolidatedsummaries. In this manner, the collectors 106 can retain a completedataset describing one period (e.g., the past minute or other suitableperiod of time), with a smaller dataset of another period (e.g., theprevious 2-10 minutes or other suitable period of time), andprogressively consolidate network traffic and corresponding data ofother periods of time (e.g., day, week, month, year, etc.). In someembodiments, network traffic and corresponding data for a set of flowsidentified as normal or routine can be winnowed at an earlier period oftime while a more complete data set may be retained for a lengthierperiod of time for another set of flows identified as anomalous or as anattack.

Computer networks may be exposed to a variety of different attacks thatexpose vulnerabilities of computer systems in order to compromise theirsecurity. Some network traffic may be associated with malicious programsor devices. The analytics engine 110 may be provided with examples ofnetwork states corresponding to an attack and network statescorresponding to normal operation. The analytics engine 110 can thenanalyze network traffic and corresponding data to recognize when thenetwork is under attack. In some embodiments, the network may operatewithin a trusted environment for a period of time so that the analyticsengine 110 can establish a baseline of normal operation. Since malwareis constantly evolving and changing, machine learning may be used todynamically update models for identifying malicious traffic patterns.

In some embodiments, the analytics engine 110 may be used to identifyobservations which differ from other examples in a dataset. For example,if a training set of example data with known outlier labels exists,supervised anomaly detection techniques may be used. Supervised anomalydetection techniques utilize data sets that have been labeled as normaland abnormal and train a classifier. In a case in which it is unknownwhether examples in the training data are outliers, unsupervised anomalytechniques may be used. Unsupervised anomaly detection techniques may beused to detect anomalies in an unlabeled test data set under theassumption that the majority of instances in the data set are normal bylooking for instances that seem to fit to the remainder of the data set.

The analytics engine 110 can include a data lake 130, an applicationdependency mapping (ADM) module 140, and elastic processing engines 150.The data lake 130 is a large-scale storage repository that providesmassive storage for various types of data, enormous processing power,and the ability to handle nearly limitless concurrent tasks or jobs. Insome embodiments, the data lake 130 is implemented using the Hadoop®Distributed File System (HDFS™) from Apache® Software Foundation ofForest Hill, Md. HDFS™ is a highly scalable and distributed file systemthat can scale to thousands of cluster nodes, millions of files, andpetabytes of data. HDFS™ is optimized for batch processing where datalocations are exposed to allow computations to take place where the dataresides. HDFS™ provides a single namespace for an entire cluster toallow for data coherency in a write-once, read-many access model. Thatis, clients can only append to existing files in the node. In HDFS™,files are separated into blocks, which are typically 64 MB in size andare replicated in multiple data nodes. Clients access data directly fromdata nodes.

In some embodiments, the data mover 108 receives raw network traffic andcorresponding data from the collectors 106 and distributes or pushes thedata to the data lake 130. The data lake 130 can also receive and storeout-of-band data 114, such as statuses on power levels, networkavailability, server performance, temperature conditions, cage doorpositions, and other data from internal sources, and third party data116, such as security reports (e.g., provided by Cisco® Systems, Inc. ofSan Jose, Calif., Arbor Networks® of Burlington, Mass., Symantec® Corp.of Sunnyvale, Calif., Sophos® Group plc of Abingdon, England, Microsoft®Corp. of Seattle, Wash., Verizon® Communications, Inc. of New York,N.Y., among others), geolocation data, IP watch lists, Whois data,configuration management database (CMDB) or configuration managementsystem (CMS) as a service, and other data from external sources. Inother embodiments, the data lake 130 may instead fetch or pull rawtraffic and corresponding data from the collectors 106 and relevant datafrom the out-of-band data sources 114 and the third party data sources116. In yet other embodiments, the functionality of the collectors 106,the data mover 108, the out-of-band data sources 114, the third partydata sources 116, and the data lake 130 can be combined. Variouscombinations and configurations are possible as would be known to one ofordinary skill in the art.

Each component of the data lake 130 can perform certain processing ofthe raw network traffic data and/or other data (e.g., host data, processdata, user data, out-of-band data or third party data) to transform theraw data to a form useable by the elastic processing engines 150. Insome embodiments, the data lake 130 can include repositories for flowattributes 132, host and/or endpoint attributes 134, process attributes136, and policy attributes 138. In some embodiments, the data lake 130can also include repositories for VM or container attributes,application attributes, tenant attributes, network topology, applicationdependency maps, cluster attributes, etc.

The flow attributes 132 relate to information about flows traversing thenetwork. A flow is generally one or more packets sharing certainattributes that are sent within a network within a specified period oftime. The flow attributes 132 can include packet header fields such as asource address (e.g., Internet Protocol (IP) address, Media AccessControl (MAC) address, Domain Name System (DNS) name, or other networkaddress), source port, destination address, destination port, protocoltype, class of service, among other fields. The source address maycorrespond to a first endpoint (e.g., network device, physical server,virtual partition, etc.) of the network, and the destination address maycorrespond to a second endpoint, a multicast group, or a broadcastdomain. The flow attributes 132 can also include aggregate packet datasuch as flow start time, flow end time, number of packets for a flow,number of bytes for a flow, the union of TCP flags for a flow, amongother flow data.

The host and/or endpoint attributes 134 describe host and/or endpointdata for each flow, and can include host and/or endpoint name, networkaddress, operating system, CPU usage, network usage, disk space, ports,logged users, scheduled jobs, open files, and information regardingfiles and/or directories stored on a host and/or endpoint (e.g.,presence, absence, or modifications of log files, configuration files,device special files, or protected electronic information). Asdiscussed, in some embodiments, the host and/or endpoints attributes 134can also include the out-of-band data 114 regarding hosts such as powerlevel, temperature, and physical location (e.g., room, row, rack, cagedoor position, etc.) or the third party data 116 such as whether a hostand/or endpoint is on an IP watch list or otherwise associated with asecurity threat, Whois data, or geocoordinates. In some embodiments, theout-of-band data 114 and the third party data 116 may be associated byprocess, user, flow, or other more granular or less granular networkelement or network communication.

The process attributes 136 relate to process data corresponding to eachflow, and can include process name (e.g., bash, httpd, netstat, etc.),ID, parent process ID, path (e.g., /usr2/username/bin/, /usr/local/bin,/usr/bin, etc.), CPU utilization, memory utilization, memory address,scheduling information, nice value, flags, priority, status, start time,terminal type, CPU time taken by the process, the command that startedthe process, and information regarding a process owner (e.g., user name,ID, user's real name, e-mail address, user's groups, terminalinformation, login time, expiration date of login, idle time, andinformation regarding files and/or directories of the user).

The policy attributes 138 contain information relating to networkpolicies. Policies establish whether a particular flow is allowed ordenied by the network as well as a specific route by which a packettraverses the network. Policies can also be used to mark packets so thatcertain kinds of traffic receive differentiated service when used incombination with queuing techniques such as those based on priority,fairness, weighted fairness, token bucket, random early detection, roundrobin, among others. The policy attributes 138 can include policystatistics such as a number of times a policy was enforced or a numberof times a policy was not enforced. The policy attributes 138 can alsoinclude associations with network traffic data. For example, flows foundto be non-conformant can be linked or tagged with corresponding policiesto assist in the investigation of non-conformance.

The analytics engine 110 may include any number of engines 150,including for example, a flow engine 152 for identifying flows (e.g.,flow engine 152) or an attacks engine 154 for identify attacks to thenetwork. In some embodiments, the analytics engine can include aseparate distributed denial of service (DDoS) attack engine 155 forspecifically detecting DDoS attacks. In other embodiments, a DDoS attackengine may be a component or a sub-engine of a general attacks engine.In some embodiments, the attacks engine 154 and/or the DDoS engine 155can use machine learning techniques to identify security threats to anetwork. For example, the attacks engine 154 and/or the DDoS engine 155can be provided with examples of network states corresponding to anattack and network states corresponding to normal operation. The attacksengine 154 and/or the DDoS engine 155 can then analyze network trafficdata to recognize when the network is under attack. In some embodiments,the network can operate within a trusted environment for a time toestablish a baseline for normal network operation for the attacks engine154 and/or the DDoS.

The analytics engine 110 may further include a search engine 156. Thesearch engine 156 may be configured, for example to perform a structuredsearch, an NLP (Natural Language Processing) search, or a visual search.Data may be provided to the engines from one or more processingcomponents.

The analytics engine 110 can also include a policy engine 158 thatmanages network policy, including creating and/or importing policies,monitoring policy conformance and non-conformance, enforcing policy,simulating changes to policy or network elements affecting policy, amongother policy-related tasks.

The ADM module 140 can determine dependencies of applications of thenetwork. That is, particular patterns of traffic may correspond to anapplication, and the interconnectivity or dependencies of theapplication can be mapped to generate a graph for the application (i.e.,an application dependency mapping). In this context, an applicationrefers to a set of networking components that provides connectivity fora given set of workloads. For example, in a conventional three-tierarchitecture for a web application, first endpoints of the web tier,second endpoints of the application tier, and third endpoints of thedata tier make up the web application. The ADM module 140 can receiveinput data from various repositories of the data lake 130 (e.g., theflow attributes 132, the host and/or endpoint attributes 134, theprocess attributes 136, etc.). The ADM module 140 may analyze the inputdata to determine that there is first traffic flowing between externalendpoints on port 80 of the first endpoints corresponding to HypertextTransfer Protocol (HTTP) requests and responses. The input data may alsoindicate second traffic between first ports of the first endpoints andsecond ports of the second endpoints corresponding to application serverrequests and responses and third traffic flowing between third ports ofthe second endpoints and fourth ports of the third endpointscorresponding to database requests and responses. The ADM module 140 maydefine an ADM for the web application as a three-tier applicationincluding a first EPG comprising the first endpoints, a second EPGcomprising the second endpoints, and a third EPG comprising the thirdendpoints.

The presentation module 116 can include an application programminginterface (API) or command line interface (CLI) 160, a securityinformation and event management (SIEM) interface 162, and a webfront-end 164. As the analytics engine 110 processes network traffic andcorresponding data and generates analytics data, the analytics data maynot be in a human-readable form or it may be too voluminous for a userto navigate. The presentation module 116 can take the analytics datagenerated by analytics engine 110 and further summarize, filter, andorganize the analytics data as well as create intuitive presentationsfor the analytics data.

In some embodiments, the API or CLI 160 can be implemented using Hadoop®Hive from Apache® for the back end, and Java® Database Connectivity(JDBC) from Oracle® Corporation of Redwood Shores, Calif., as an APIlayer. Hive is a data warehouse infrastructure that provides datasummarization and ad hoc querying. Hive provides a mechanism to querydata using a variation of structured query language (SQL) that is calledHiveQL. JDBC is an application programming interface (API) for theprogramming language Java®, which defines how a client may access adatabase.

In some embodiments, the SIEM interface 162 can be implemented usingHadoop® Kafka for the back end, and software provided by Splunk®, Inc.of San Francisco, Calif. as the SIEM platform. Kafka is a distributedmessaging system that is partitioned and replicated. Kafka uses theconcept of topics. Topics are feeds of messages in specific categories.In some embodiments, Kafka can take raw packet captures and telemetryinformation from the data mover 108 as input, and output messages to aSIEM platform, such as Splunk®. The Splunk® platform is utilized forsearching, monitoring, and analyzing machine-generated data.

In some embodiments, the web front-end 164 can be implemented usingsoftware provided by MongoDB®, Inc. of New York, N.Y. and Hadoop®ElasticSearch from Apache® for the back-end, and Ruby on Rails™ as theweb application framework. MongoDB® is a document-oriented NoSQLdatabase based on documents in the form of JavaScript® Object Notation(JSON) with dynamic schemas. ElasticSearch is a scalable and real-timesearch and analytics engine that provides domain-specific language (DSL)full querying based on JSON. Ruby on Rails™ is model-view-controller(MVC) framework that provides default structures for a database, a webservice, and web pages. Ruby on Rails™ relies on web standards such asJSON or extensible markup language (XML) for data transfer, andhypertext markup language (HTML), cascading style sheets, (CSS), andJavaScript® for display and user interfacing.

Although FIG. 1 illustrates an example configuration of the variouscomponents of a network traffic monitoring system, those of skill in theart will understand that the components of the network trafficmonitoring system 100 or any system described herein can be configuredin a number of different ways and can include any other type and numberof components. For example, the sensors 104, the collectors 106, thedata mover 108, and the data lake 130 can belong to one hardware and/orsoftware module or multiple separate modules. Other modules can also becombined into fewer components and/or further divided into morecomponents.

FIG. 2 illustrates an example of a network environment 200 in accordancewith an embodiment. In some embodiments, a network traffic monitoringsystem, such as the network traffic monitoring system 100 of FIG. 1, canbe implemented in the network environment 200. It should be understoodthat, for the network environment 200 and any environment discussedherein, there can be additional or fewer nodes, devices, links,networks, or components in similar or alternative configurations.Embodiments with different numbers and/or types of clients, networks,nodes, cloud components, servers, software components, devices, virtualor physical resources, configurations, topologies, services, appliances,deployments, or network devices are also contemplated herein. Further,the network environment 200 can include any number or type of resources,which can be accessed and utilized by clients or tenants. Theillustrations and examples provided herein are for clarity andsimplicity.

The network environment 200 can include a network fabric 202, a Layer 2(L2) network 204, a Layer 3 (L3) network 206, and servers 208 a, 208 b,208 c, 208 d, and 208 e (collectively, 208). The network fabric 202 caninclude spine switches 210 a, 210 b, 210 c, and 210 d (collectively,“210”) and leaf switches 212 a, 212 b, 212 c, 212 d, and 212 e(collectively, “212”). The spine switches 210 can connect to the leafswitches 212 in the network fabric 202. The leaf switches 212 caninclude access ports (or non-fabric ports) and fabric ports. The fabricports can provide uplinks to the spine switches 210, while the accessports can provide connectivity to endpoints (e.g., the servers 208),internal networks (e.g., the L2 network 204), or external networks(e.g., the L3 network 206).

The leaf switches 212 can reside at the edge of the network fabric 202,and can thus represent the physical network edge. For instance, in someembodiments, the leaf switches 212 d and 212 e operate as border leafswitches in communication with edge devices 214 located in the externalnetwork 206. The border leaf switches 212 d and 212 e may be used toconnect any type of external network device, service (e.g., firewall,deep packet inspector, traffic monitor, load balancer, etc.), or network(e.g., the L3 network 206) to the fabric 202.

Although the network fabric 202 is illustrated and described herein asan example leaf-spine architecture, one of ordinary skill in the artwill readily recognize that various embodiments can be implemented basedon any network topology, including any data center or cloud networkfabric. Indeed, other architectures, designs, infrastructures, andvariations are contemplated herein. For example, the principlesdisclosed herein are applicable to topologies including three-tier(including core, aggregation, and access levels), fat tree, mesh, bus,hub and spoke, etc. Thus, in some embodiments, the leaf switches 212 canbe top-of-rack switches configured according to a top-of-rackarchitecture. In other embodiments, the leaf switches 212 can beaggregation switches in any particular topology, such as end-of-row ormiddle-of-row topologies. In some embodiments, the leaf switches 212 canalso be implemented using aggregation switches.

Moreover, the topology illustrated in FIG. 2 and described herein isreadily scalable and may accommodate a large number of components, aswell as more complicated arrangements and configurations. For example,the network may include any number of fabrics 202, which may begeographically dispersed or located in the same geographic area. Thus,network nodes may be used in any suitable network topology, which mayinclude any number of servers, virtual machines or containers, switches,routers, appliances, controllers, gateways, or other nodesinterconnected to form a large and complex network. Nodes may be coupledto other nodes or networks through one or more interfaces employing anysuitable wired or wireless connection, which provides a viable pathwayfor electronic communications.

Network communications in the network fabric 202 can flow through theleaf switches 212. In some embodiments, the leaf switches 212 canprovide endpoints (e.g., the servers 208), internal networks (e.g., theL2 network 204), or external networks (e.g., the L3 network 206) accessto the network fabric 202, and can connect the leaf switches 212 to eachother. In some embodiments, the leaf switches 212 can connect endpointgroups (EPGs) to the network fabric 202, internal networks (e.g., the L2network 204), and/or any external networks (e.g., the L3 network 206).EPGs are groupings of applications, or application components, and tiersfor implementing forwarding and policy logic. EPGs can allow forseparation of network policy, security, and forwarding from addressingby using logical application boundaries. EPGs can be used in the networkenvironment 200 for mapping applications in the network. For example,EPGs can comprise a grouping of endpoints in the network indicatingconnectivity and policy for applications.

As discussed, the servers 208 can connect to the network fabric 202 viathe leaf switches 212. For example, the servers 208 a and 208 b canconnect directly to the leaf switches 212 a and 212 b, which can connectthe servers 208 a and 208 b to the network fabric 202 and/or any of theother leaf switches. The servers 208 c and 208 d can connect to the leafswitches 212 b and 212 c via the L2 network 204. The servers 208 c and208 d and the L2 network 204 make up a local area network (LAN). LANscan connect nodes over dedicated private communications links located inthe same general physical location, such as a building or campus.

The WAN 206 can connect to the leaf switches 212 d or 212 e via the L3network 206. WANs can connect geographically dispersed nodes overlong-distance communications links, such as common carrier telephonelines, optical light paths, synchronous optical networks (SONET), orsynchronous digital hierarchy (SDH) links. LANs and WANs can include L2and/or L3 networks and endpoints.

The Internet is an example of a WAN that connects disparate networksthroughout the world, providing global communication between nodes onvarious networks. The nodes typically communicate over the network byexchanging discrete frames or packets of data according to predefinedprotocols, such as the Transmission Control Protocol/Internet Protocol(TCP/IP). In this context, a protocol can refer to a set of rulesdefining how the nodes interact with each other. Computer networks maybe further interconnected by an intermediate network node, such as arouter, to extend the effective size of each network. The endpoints 208can include any communication device or component, such as a computer,server, blade, hypervisor, virtual machine, container, process (e.g.,running on a virtual machine), switch, router, gateway, host, device,external network, etc.

In some embodiments, the network environment 200 also includes a networkcontroller running on the host 208 a. The network controller isimplemented using the Application Policy Infrastructure Controller(APIC™) from Cisco®. The APIC™ provides a centralized point ofautomation and management, policy programming, application deployment,and health monitoring for the fabric 202. In some embodiments, the APIC™is operated as a replicated synchronized clustered controller. In otherembodiments, other configurations or software-defined networking (SDN)platforms can be utilized for managing the fabric 202.

In some embodiments, a physical server 208 may have instantiated thereona hypervisor 216 for creating and running one or more virtual switches(not shown) and one or more virtual machines 218, as shown for the host208 b. In other embodiments, physical servers may run a shared kernelfor hosting containers. In yet other embodiments, the physical server208 can run other software for supporting other virtual partitioningapproaches. Networks in accordance with various embodiments may includeany number of physical servers hosting any number of virtual machines,containers, or other virtual partitions. Hosts may also compriseblade/physical servers without virtual machines, containers, or othervirtual partitions, such as the servers 208 a, 208 c, 208 d, and 208 e.

The network environment 200 can also integrate a network trafficmonitoring system, such as the network traffic monitoring system 100shown in FIG. 1. For example, the network traffic monitoring system ofFIG. 2 includes sensors 220 a, 220 b, 220 c, and 220 d (collectively,“220”), collectors 222, and an analytics engine, such as the analyticsengine 110 of FIG. 1, executing on the server 208 e. The analyticsengine 208 e can receive and process network traffic data collected bythe collectors 222 and detected by the sensors 220 placed on nodeslocated throughout the network environment 200. Although the analyticsengine 208 e is shown to be a standalone network appliance in FIG. 2, itwill be appreciated that the analytics engine 208e can also beimplemented as a virtual partition (e.g., VM or container) that can bedistributed onto a host or cluster of hosts, software as a service(SaaS), or other suitable method of distribution. In some embodiments,the sensors 220 run on the leaf switches 212 (e.g., the sensor 220 a),the hosts 208 (e.g., the sensor 220 b), the hypervisor 216 (e.g., thesensor 220 c), and the VMs 218 (e.g., the sensor 220 d). In otherembodiments, the sensors 220 can also run on the spine switches 210,virtual switches, service appliances (e.g., firewall, deep packetinspector, traffic monitor, load balancer, etc.) and in between networkelements. In some embodiments, sensors 220 can be located at each (ornearly every) network component to capture granular packet statisticsand data at each hop of data transmission. In other embodiments, thesensors 220 may not be installed in all components or portions of thenetwork (e.g., shared hosting environment in which customers haveexclusive control of some virtual machines).

As shown in FIG. 2, a host may include multiple sensors 220 running onthe host (e.g., the host sensor 220 b) and various components of thehost (e.g., the hypervisor sensor 220 c and the VM sensor 220 d) so thatall (or substantially all) packets traversing the network environment200 may be monitored. For example, if one of the VMs 218 running on thehost 208 b receives a first packet from the WAN 206, the first packetmay pass through the border leaf switch 212 d, the spine switch 210 b,the leaf switch 212 b, the host 208 b, the hypervisor 216, and the VM.Since all or nearly all of these components contain a respective sensor,the first packet will likely be identified and reported to one of thecollectors 222. As another example, if a second packet is transmittedfrom one of the VMs 218 running on the host 208 b to the host 208 d,sensors installed along the data path, such as at the VM 218, thehypervisor 216, the host 208 b, the leaf switch 212 b, and the host 208d will likely result in capture of metadata from the second packet.

FIG. 3 illustrates an example of a data pipeline 300 for determiningclusters in an application dependency map in accordance with an exampleembodiment. In some embodiments, the data pipeline 300 can be directedby a network traffic monitoring system, such as the network trafficmonitoring system 100 of FIG. 1; an analytics engine, such as theanalytics engine 110 of FIG. 1; an application dependency mappingmodule, such as the ADM module 140 of FIG. 1; or other network serviceor network appliance. The data pipeline 300 includes a data collectionstage 302 in which network traffic data and corresponding data (e.g.,host data, process data, user data, etc.) are captured by sensors (e.g.,the sensors 104 of FIG. 1) located throughout the network. The data maycomprise, for example, raw flow data and raw process data. As discussed,the data can be captured from multiple perspectives to provide acomprehensive view of the network. The data collected may also includeother types of information, such as tenant information, virtualpartition information, out-of-band information, third party information,and other relevant information. In some embodiments, the flow data andassociated data can be aggregated and summarized daily or according toanother suitable increment of time, and flow vectors, process vectors,host vectors, and other attribute vectors can be calculated during thedata collection stage 302. This can substantially reduce processingduring an ADM run.

The data pipeline 300 also includes an ADM input data stage 304 in whicha network or security administrator or other authorized user mayconfigure an ADM run by selecting the date range of the flow data andassociated data to analyze, and those nodes for which the administratorwants application dependency maps and/or cluster information. In someembodiments, the administrator can also input side information, such asserver load balance, route tags, and previously identified clustersduring the ADM input data stage 304. In other embodiments, the sideinformation can be automatically pulled or another network element canpush the side information for the ADM run.

The next stage of the data pipeline 300 is pre-processing 306. Duringthe pre-processing stage 306, nodes of the network are partitioned intoselected node and dependency node subnets. Selected nodes are thosenodes for which the user requests application dependency maps andcluster information. Dependency nodes are those nodes that are notexplicitly selected by the users for an ADM run but are nodes thatcommunicate with the selected nodes. To obtain the partitioninginformation, edges of an application dependency map (i.e., flow data)and unprocessed attribute vectors can be analyzed.

Other tasks can also be performed during the pre-processing stage 306,including identifying dependencies of the selected nodes and thedependency nodes; replacing the dependency nodes with tags based on thedependency nodes' subnet names; extracting attribute vectors for theselected nodes, such as by aggregating daily vectors across multipledays, calculating term frequency-inverse document frequency (tf-idf),and normalizing the vectors (e.g., l₂ normalization); and identifyingexisting clusters.

In some embodiments, the pre-processing stage 306 can include earlyattribute fusion pre-processing. Early fusion is a fusion scheme inwhich attributes are combined into a single representation. Attributesmay be derived from various domains (e.g., network, host, virtualpartition, process, user, etc.), and an attribute vector in an earlyfusion system may represent the concatenation of disparate attributetypes or domains.

Early fusion may be effective for attributes that are similar or have asimilar structure (e.g., fields of TCP and UDP packets or flows). Suchattributes may be characterized as being a same type or being within asame domain. Early fusion may be less effective for distant attributesor attributes of different types or domains (e.g., flow-based attributesversus process-based attributes). Thus, in some embodiments, onlyattributes in the network domain (i.e., network traffic-basedattributes, such as packet header information, number of packets for aflow, number of bytes for a flow, and similar data) may be analyzed. Inother embodiments, an ADM run may limit analysis to attributes in theprocess domain (i.e., process-based attributes, such as process name,parent process, process owner, etc.). In yet other embodiments,attribute sets in other domains (e.g., the host domain, virtualpartition domain, user domain, etc.) may be the focus of the ADM run.

After pre-processing, the data pipeline 300 may proceed to a clusteringstage 308. In the clustering stage 308, various machine learningtechniques can be implemented to analyze attribute vectors within asingle domain or across different domains to determine the optimalclustering given a set of input nodes. Machine learning is an area ofcomputer science in which the goal is to develop models using exampleobservations (i.e., training data), that can be used to make predictionson new observations. The models or logic are not based on theory but areempirically based or data-driven.

During the clustering stage 308, respective attribute vectors of nodesare evaluated using machine learning to identify an optimal clusteringfor a selected set of nodes. Supervised or unsupervised learningtechniques can be used depending on the availability of training dataand other related information (e.g., network topology). For example, anADM module (or other suitable system) can receive configurationinformation regarding a network from a configuration management system(CMS), configuration management database (CMDB), or other similarsystem. In some embodiments, the ADM module can receive theconfiguration data in a proprietary or open source format utilized bythe CMS or CMDB and translate the information to training dataobservations for the particular machine learning approach(es)implemented by the ADM module. In other embodiments, the CMS or CMDB andthe ADM module may be closely integrated and the CMS or CMDB canautomatically provide the ADM module with suitable training data. In yetother embodiments, a network administrator or authorized user mayreceive the configuration data from the CM and the administrator or usercan manually label nodes to create the training data.

In some embodiments network traffic monitoring system 100 is useful forpresenting a visualization of data flows so that a network administratorcan better monitor or investigation data flows. Presentation module 112can present an application dependency map by providing a dynamic graphinterface effective to customize a view of an application.

Multi-tier applications can be very complex. Thousands of logicalentities could potentially be responsible for the performance of anapplication. Given their complexity, a multi-tier application cannoteasily be viewed in a conventional dependency graph.

FIG. 4 illustrates an example method for producing an applicationdependency graph in a dynamic graph interface, and FIG. 5A and FIG. 5Billustrate an example graph 500 in an example graph interface andillustrate functionality provided by the dynamic graph interface.

In some embodiments, graph 500 can be produced by presentation module112 in combination with ADM engine 140 by searching data analyzed by ananalytics engine (ADM engine 140) in response to a received query tovisualize a multi-tier application 402.

The search can identify a plurality of logical entities, which couldnumber thousands or more logical entities depending on the application,and select 404 a limited number of logical entities to represent themulti-tier application based on the query criteria. It would not beuseful to represent the application in great detail, as the amount ofinformation presented would be overwhelming and hard to visualize in thedynamic graph interface. Instead, presentation module 112 can beconfigured to only represent a maximum number of nodes representinglogical entities in an initially presented graph. The maximum number canbe configurable. Likewise a minimum number of nodes representing logicalentities can also be configured.

The selected 404 logical entities are used to represent 406 themulti-tier application in a dynamic graph 500 as a limited number ofnodes representing logical entities. As illustrated in FIG. 5A, tennodes are used to represent the application in graph 500. However, asillustrated in FIG. 5B, which expands only node 508, many additionallogical entities are collapsed into clusters of logical entities. Thelogical entities within the cluster can be many layers deep, or canrepresent endpoints. The logical entities/clusters can be determined bythe ADM engine 140, as described above.

In some embodiments, the limited number of nodes representing logicalentities can be arranged 408 based on the query criteria. While somequeries might be higher level queries that request to view an entireapplication, some queries might be lower level queries that attempt toview a tier of the multi-tier application, or entities grouped into agiven subnet, or functional entities. In some embodiments, the querymight even identify a particular logical entity that is desired to beviewed. Especially in embodiments wherein the query has some specificityit can be desirable to arrange the logical entities (and select entitiesto be displayed) based on search criteria. For example, if a cluster oflogical entities best matches the search criteria, a nodes representingthat cluster might be arranged into the center of the dynamic graph, orarranged to the left of the dynamic graph, or in any arrangementselected by an arrangement algorithm. In such embodiments, datareflecting how well a logical entity matches search criteria can be aninput into a graph layout algorithm.

Since multi-tier applications can be some complex and have so manylogical entities, it is a benefit of the present technology to providefunctionality that allows the dynamic graph interface to receive inputseffective to customize the graph, or to explore the graph.

FIG. 5A illustrates an example initial view of a graph 500 in thedynamic graph interface. Graph 500 presents logical entities ofapplication 502 “nprd.” The graph presents a plurality of nodes such asnode 508 and node 514. Some nodes are fully expanded such as node 514,which is represented as a filled in circle. While collapsed nodes arevisually district, such as node 508, which is represented as an outlinedcircle. Lines between the nodes can represent edges of graph, or logicalcommunication flows.

In some embodiments, the dynamic graph interface can receive an input,such as a double click, on one of the nodes, and in response to theinput can the graph interface can explode 410 the node in to a pluralityof additional logical entities. This is illustrated in FIG. 5, whereinnode 508 in FIG. 5A is illustrated as an outlined circle—indicating thatit is a node representing a collection of collapsed logical entities.When a double click is received on node 508, it can become expanded asillustrated in FIG. 5B. In FIG. 5B node 508 has now become a filled incircle indicating that it is completely expanded. However, the nodesrepresenting the logical entities that have expanded from node 508 mayrepresent further logical entities. See for example, node 524, which isrepresented as an outlined circle; this indicates it too can beexpanded.

As a node 410 is expanded, the graph 500 can be rearranged 412 toaccommodate the plurality of additional nodes representing logicalentities. As illustrated in FIG. 5B some of the already existing nodesin graph 500 have shifted to the right, and some nodes have adjustedtheir spacing.

While the expansion of node 508 appears to have expanded into atree-like structure, it is important to note that the actual structureof the graph is not a tree. The expanded nodes representing logicalentities does not necessary represent end nodes, and in many cases theexpanded nodes representing logical entities might intercommunicate orcommunicate back to other existing entities in the graph. However, insome embodiments, it can be useful to give a tree-like impression to theuser of the dynamic graph interface. If a user is looking to see wherecommunications flow from entity 508, it may be most comfortable to theuser to view the expanded entities to the right, which appear to bedownstream. However, this is just an illusion created for convenience.None of the expanded entities need to be downstream of entity 508.

In addition to the expansion of logical entities, FIG. 5A illustratesadditional inputs available for arranging and rearranging graph 500. Thegraph can be zoomed in/out with controls 504. Additionally, nodesrepresenting logical entities can be anchored in place with control 506.Nodes 514 and 508 have been anchored in place as represented by ananchor symbol inside the circle representing the entity. When a graph isrearranged, the nodes anchored in place will not move. In someembodiments, graph 500 can be rearranged to make a node representing anentity the center of the graph 500 using control 510.

In addition to the illustrated controls, nodes representing logicalentities can be repositioned through drag actions received in thedynamic graph interface. Nodes representing logical entities can also behidden or collapsed into other nodes.

When a node, such as node 508 in FIG. 5A, that represents a collectionof other entities is expanded, the node can transform to represent onlya single entity, such as node 508 in FIG. 5B. This can be evidentthrough information display 516 in FIG. 5A and FIG. 5B. When a node isselected, information regarding that node can be displayed ininformation display 516. For example in FIG. 5A, node 508 is selected,and information display 516 shows the name of the logical entityrepresented by the node, the number of endpoints represented by thenode, the number of neighbors to the logical entities represented by thenode, the number of subnets encompassed by the node, the number oflogical entities that are provided with data or communication flows fromentities represented by the node, and the number of logical entitiesthat consume data or communication flows from entities represented bythe node. In FIG. 5B, the information regarding the node has changedbecause node 508 has been expanded and many of the entities previouslyrepresented by node 508 are represented by other nodes (such as node524).

Once graph 500 is arranged in a way suitable to the user, control 512can be used to save and share the graph. In some embodiments, thearrangement of the graph can be saved as a template for futurevisualizations of the same application with updated data.

FIG. 6 illustrates a conventional system bus computing systemarchitecture 600 that can be used with any of the system componentsillustrated in FIG. 1 wherein the components of the system 600 are inelectrical communication with each other using a bus 605. Exemplarysystem 600 includes a processing unit (CPU or processor) 610 and asystem bus 605 that couples various system components including thesystem memory 615, such as read only memory (ROM) 670 and random accessmemory (RAM) 675, to the processor 610. The system 600 can include acache of high-speed memory connected directly with, in close proximityto, or integrated as part of the processor 610. The system 600 can copydata from the memory 615 and/or the storage device 630 to the cache 612for quick access by the processor 610. In this way, the cache canprovide a performance boost that avoids processor 610 delays whilewaiting for data. These and other modules can control or be configuredto control the processor 610 to perform various actions. Other systemmemory 615 may be available for use as well. The memory 615 can includemultiple different types of memory with different performancecharacteristics. The processor 610 can include any general purposeprocessor and a hardware module or software module, such as module 1637, module 2 634, and module 3 636 stored in storage device 630,configured to control the processor 610 as well as a special-purposeprocessor where software instructions are incorporated into the actualprocessor design. The processor 610 may essentially be a completelyself-contained computing system, containing multiple cores orprocessors, a bus, memory controller, cache, etc. A multi-core processormay be symmetric or asymmetric.

To enable user interaction with the computing device 600, an inputdevice 645 can represent any number of input mechanisms, such as amicrophone for speech, a touch-sensitive screen for gesture or graphicalinput, keyboard, mouse, motion input, speech and so forth. An outputdevice 635 can also be one or more of a number of output mechanismsknown to those of skill in the art. In some instances, multimodalsystems can enable a user to provide multiple types of input tocommunicate with the computing device 600. The communications interface640 can generally govern and manage the user input and system output.There is no restriction on operating on any particular hardwarearrangement and therefore the basic attributes here may easily besubstituted for improved hardware or firmware arrangements as they aredeveloped.

Storage device 630 is a non-volatile memory and can be a hard disk orother types of computer readable media which can store data that areaccessible by a computer, such as magnetic cassettes, flash memorycards, solid state memory devices, digital versatile disks, cartridges,random access memories (RAMs) 675, read only memory (ROM) 680, andhybrids thereof.

The storage device 630 can include software modules 638, 634, 636 forcontrolling the processor 610. Other hardware or software modules arecontemplated. The storage device 630 can be connected to the system bus605. In one aspect, a hardware module that performs a particularfunction can include the software component stored in acomputer-readable medium in connection with the necessary hardwarecomponents, such as the processor 610, bus 605, display 635, and soforth, to carry out the function.

For clarity of explanation, in some instances the present technology maybe presented as including individual functional blocks includingfunctional blocks comprising devices, device components, steps orroutines in a method embodied in software, or combinations of hardwareand software.

In some embodiments the computer-readable storage devices, mediums, andmemories can include a cable or wireless signal containing a bit streamand the like. However, when mentioned, non-transitory computer-readablestorage media expressly exclude media such as energy, carrier signals,electromagnetic waves, and signals per se.

Methods according to the above-described examples can be implementedusing computer-executable instructions that are stored or otherwiseavailable from computer readable media. Such instructions can comprise,for example, instructions and data which cause or otherwise configure ageneral purpose computer, special purpose computer, or special purposeprocessing device to perform a certain function or group of functions.Portions of computer resources used can be accessible over a network.The computer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, firmware, orsource code. Examples of computer-readable media that may be used tostore instructions, information used, and/or information created duringmethods according to described examples include magnetic or opticaldisks, flash memory, USB devices provided with non-volatile memory,networked storage devices, and so on.

Devices implementing methods according to these disclosures can comprisehardware, firmware and/or software, and can take any of a variety ofform factors. Typical examples of such form factors include laptops,smart phones, small form factor personal computers, personal digitalassistants, rackmount devices, standalone devices, and so on.Functionality described herein also can be embodied in peripherals oradd-in cards. Such functionality can also be implemented on a circuitboard among different chips or different processes executing in a singledevice, by way of further example.

The instructions, media for conveying such instructions, computingresources for executing them, and other structures for supporting suchcomputing resources are means for providing the functions described inthese disclosures.

Although a variety of examples and other information was used to explainaspects within the scope of the appended claims, no limitation of theclaims should be implied based on particular attributes or arrangementsin such examples, as one of ordinary skill would be able to use theseexamples to derive a wide variety of implementations. Further andalthough some subject matter may have been described in languagespecific to examples of structural attributes and/or method steps, it isto be understood that the subject matter defined in the appended claimsis not necessarily limited to these described attributes or acts. Forexample, such functionality can be distributed differently or performedin components other than those identified herein. Rather, the describedattributes and steps are disclosed as examples of components of systemsand methods within the scope of the appended claims. Moreover, claimlanguage reciting “at least one of” a set indicates that one member ofthe set or multiple members of the set satisfy the claim.

1. A method for visualizing a multi-tier application comprising:representing the multi-tier application in a dynamic graph as a limitednumber of logical entities, at least one logical entity being a firstcluster of additional logical entities; receiving an input effective toexplode the first cluster of logical entities in to a plurality ofadditional logical entities, at least one of the additional logicalentities being a second cluster of logical entities; and rearranging thedynamic graph to accommodate the plurality of additional logicalentities.
 2. The method of claim 1, comprising: selecting the limitednumber of logical entities to represent the multi-tier application basedon query criteria, and arranging the limited number of logical entitiesbased on the query criteria.
 3. The method of claim 1, comprising:representing logical entities being a cluster of additional logicalentities in a first representation, while representing logical entitiesthat have been fully exploded to reveal all dependencies in a secondrepresentation.
 4. The method of claim 1, wherein logical entities thatare partially exploded such that some dependencies are revealed whileother dependencies are clustered are represented in the firstrepresentation.
 5. The method of claim 1, comprising: anchoring alogical entity in response to receiving an input, wherein when thedynamic graph is rearranged, the logical entity remains anchored inplace.
 6. The method of claim 1, comprising: receiving a selection ofone of the plurality of logical entities; and presenting a descriptionof the logical entity.
 7. The method of claim 5, wherein the rearrangingthe dynamic graph to accommodate the plurality of additional logicalentities includes shifting the existing logical entities to the left,except the anchored logical entity, and introducing the additionallogical entities to the right
 8. The method of claim 1, comprising:hiding a logical entity in response to a received input.
 9. A system forarranging a graph representing a multi-tier application the systemcomprising: a processor; and a non-transitory computer readable mediumstoring processor executable instructions, the instructions effective tocause the processor to: represent the multi-tier application in adynamic graph as a limited number of nodes representing logicalentities, at least one node being a first cluster of additional logicalentities; receive an input effective to explode the first cluster oflogical entities in to a plurality of additional nodes representinglogical entities, at least one of the additional nodes being a secondcluster of logical entities; and rearranging the dynamic graph toaccommodate the plurality of additional nodes representing logicalentities.
 10. The system of claim 9, wherein the instructions areeffective to: select the nodes representing logical entities torepresent the multi-tier application based on query criteria, andarrange the nodes representing logical entities based on the querycriteria.
 11. The system of claim 9, wherein the instructions areeffective to: represent nodes being a cluster of additional logicalentities in a first representation, while representing nodes that havebeen fully exploded to reveal all dependencies in a secondrepresentation.
 12. The system of claim 9, wherein nodes are partiallyexploded such that some dependencies are revealed while otherdependencies are clustered are represented in the first representation.13. The system of claim 9, wherein the instructions are effective to:anchor a node in response to receiving an input, wherein when thedynamic graph is rearranged, the node remains anchored in place.
 14. Thesystem of claim 9, wherein the instructions are effective to: receive aselection of one of nodes; and present a description of the logicalentity represented by the node.
 15. A non-transitory computer readablemedium comprising instructions stored thereon, the instructionseffective to cause the processor to: represent the multi-tierapplication in a dynamic graph as a limited number of nodes representinglogical entities, at least one node being a first cluster of additionallogical entities; receive an input effective to explode the firstcluster of logical entities in to a plurality of additional nodesrepresenting logical entities, at least one of the additional nodesbeing a second cluster of logical entities; and rearranging the dynamicgraph to accommodate the plurality of additional nodes representinglogical entities.
 16. The non-transitory computer readable medium ofclaim 15, wherein the instructions are effective to: select the nodesrepresenting logical entities to represent the multi-tier applicationbased on query criteria, and arrange the nodes representing logicalentities based on the query criteria.
 17. The non-transitory computerreadable medium of claim 15, wherein the instructions are effective to:represent nodes being a cluster of additional logical entities in afirst representation, while representing nodes that have been fullyexploded to reveal all dependencies in a second representation.
 18. Thenon-transitory computer readable medium of claim 15, wherein nodes arepartially exploded such that some dependencies are revealed while otherdependencies are clustered are represented in the first representation.19. The non-transitory computer readable medium of claim 15, wherein theinstructions are effective to: anchor a node in response to receiving aninput, wherein when the dynamic graph is rearranged, the node remainsanchored in place.
 20. The non-transitory computer readable medium ofclaim 15, wherein the instructions are effective to: receive an inputeffective to explode the first cluster of logical entities in to aplurality of additional nodes representing logical entities, at leastone of the additional nodes being a second cluster of logical entities,wherein the expanded additional nodes are presented to give an illusionof a tree structure.